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Abstract — Orthogonal Matching Pursuit (OMP) is a canonical 
greedy pursuit algorithm for sparse approximation. Previous 
studies of OMP have mainly considered the exact recovery of 
a sparse signal x through <1? and y = 3?x, where <& is a matrix 
with more columns than rows. In this paper, based on Restricted 
Isometry Property (RIP), the performance of OMP is analyzed 
under general perturbations, which means both y and <& are 
perturbed. Though exact recovery of an almost sparse signal 
x is no longer feasible, the main contribution reveals that the 
exact recovery of the locations of k largest magnitude entries 
of x can be guaranteed under reasonable conditions. The error 
between x and solution of OMP is also estimated. It is also 
demonstrated that the sufficient condition is rather tight by 
constructing an example. When x is strong-decaying, it is proved 
that the sufficient conditions can be relaxed, and the locations 
can even be recovered in the order of the entries' magnitude. 

Index Terms — Orthogonal Matching Pursuit (OMP), Re- 
stricted Isometry Property (RIP), Compressed Sensing (CS), 
support recovery, strong-decaying signals, general perturba- 
tions. 



I. Introduction 

FINDING the sparse solution of underdetermined linear 
equation 

y = $x (1) 

is one of the basic problems in some fields of signal process- 
ing, where y E C m and e £mxn w ^t;h m < n. The basic 
problem ([TJ has arisen in many applications, including Sparse 
Component Analysis (SCA) fT), E) and Blind Source Sepa- 
ration (BSS) Q, |4j. Since the introduction of Compressed 
Sensing (CS) 0-ED, the problem (Q~|) has received significant 
attention in the past decade. In the field of CS, y denotes the 
measurement vector, is called the sensing matrix, and x is 
the sparse or almost sparse signal to be recovered. 

Various algorithms have been proposed to recover x. They 
roughly fall into two categories: 

Convex relaxation: Based on linear programming tech- 
nique, finding the sparsest solution to (Q]) can be relaxed to 
a convex optimization problem, also known as Basis Pursuit 
(BP) [6 |. Algorithms used to complete the optimization include 
Interior-point Methods [9|, Projected Gradient Methods |10|, 
and Iterative Thresholding ifTTl . 

Greedy pursuits: These algorithms build up an approx- 
imated set of nonzero locations by making locally optimal 
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choices in each iteration. Several popular ones are Orthogonal 
Matching Pursuit (OMP) 02-03), Regularized Orthogo- 
nal Matching Pursuit (ROMP) [15], Compressive Sampling 
Matching Pursuit (CoSaMP) 02), Subspace Pursuit (SP) 03, 
and Iterative Hard Thresholding (IHT) 03). 

For the scenario of no perturbations, the recovery process 
can be formulated as 

(N ) x = R(y, *,.-•), 

where R(-) denotes the process of a recovery algorithm, with 
the inputs listed in the following brackets, and x denotes the 
output, i.e. the approximation of the original sparse signal x. 
Process of (No) is non-perturbed, thus sparse signal can be 
exactly recovered under suitable conditions. For example, un- 
der certain conditions, BP flD, EO), OMP ED-EI), ROMP 
ifTBl CoSaMP 02) and SP Q7) all guarantee exact recovery 
of x. 

In practical applications, the measurement vector y is often 
contaminated by noise. Thus a perturbed measurement vector 
in the form of 

y = y + b (2) 

is considered, where b denotes measurement perturbation. In 
such scenario, the recovery process can be formulated as 

(JVi) x = R(y, *,..-). 

Plentiful studies of recovery algorithms including BP (19), 

EI-ED, omp El, ED, EE), EO, El, romp E3, 

CoSaMP 02), SP 03, IHT 03), and other El have con- 
sidered the recovery accuracy in (N±) process. Define the 
support set supp(-) as the set composed of the locations of 
all nonzero entries of a vector. It has been shown that OMP 
will exactly recover the support set of a sparse signal x from 
the perturbed observation vector, i.e. supp(£) = supp(a;), if 
certain requirements are satisfied with the coherence parameter 
H (Th.5.1 in ED, Th.4 in ED or Th.3.1 in E2) or Restricted 
Isometry Property (RIP) (Th.2 in ESI ). 

Existing results have mainly focused on the measurement 
perturbation, yet researches concerning the general pertur- 
bations are relatively rare. Here, the general perturbations 
involve a perturbed sensing matrix as well as a perturbed mea- 
surement vector. Two scenarios are considered in this paper 
from different perspective of views, where the measurement 
perturbation b is both involved due to imperfect measuring 
such as quantization effects. 

The first scenario is from user's perspective of view, which 
means the sensing matrix of the system is not accurately 
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known, or not known at all. Thus the sensing process is in 
the form of 

y = &x + b, = * + (3) 

with recovery process 

(N 2 ) x = R(y, *,.-.). 

The system perturbation E is introduced because of mis- 
modeling of the system, or the error involved during system 
calibration. Since the available sensing matrix is the perturbed 
$ instead of <fr, the condition for recovery is also in terms of 
the former. 

The second scenario is from designer's perspective of view, 
which means the system perturbation E is introduced by 
physical implementation of a designed system model <I> |35|. 
Thus the sensing process is in the form of 

y = $x + b, * = * + E, (4) 

with recovery process 

(N^) x = R(y, *,-..). 

Since the available sensing matrix is the ideal one, the condi- 
tion for recovery should be in terms of <fr in this scenario. 

Herman and Strohmer have studied the accuracy of BP 
solution in (N2) process ll36l . Later, Herman and Needell 
also gave the recovery error of CoSaMP |f37l . However, as 
far as we know, few works have been done yet on the perfect 
support recovery or the recovery error of OMP under general 
perturbations. 

Analysis of OMP considering general perturbations and 
support recovery may benefit the analysis of many other 
greedy algorithms. In some practical applications, recovering 
the support set other than a more accurate estimation is a 
fundamental concern (35], [38 1. In this paper, a completely 
perturbed scenario in the form of (01 is considered and the 
performance of OMP in (A/2) process is studied. It is shown 
that under certain RIP based conditions, the locations of k 
largest magnitude entries of a almost sparse signal x can be 
exactly recovered via OMP. Furthermore, an upper bound on 
the recovery error is given. It is also demonstrated that the 
results generalize the previous study concerning OMP in (No) 
process in [23|-[25|, |27|. The completely perturbed scenario 
(HJi together with (A^) process is also discussed. 

The rest of the paper is organized as follows. Section II 
gives a brief review of OMP and RIP, as well as certain 
necessary assumptions and notations. Section III presents the 
main theoretical results on the completely perturbed scenarios. 
Several extensions are also presented with respect to special 
signals. Section IV provides the proofs of the theorems. 
Section V discusses some related works. The whole paper is 
concluded in Section VI. To make the paper more readable, 
some proofs are relegated as an appendix in Section VII. 

II. Background 

A. Orthogonal Matching Pursuit (OMP) 

The key idea of OMP lies in the attempt to reconstruct 
the support set A of a; iteratively by starting with A = 0. 



TABLE I 
The OMP Algorithm 



Input: y, «I>; 

Initialization: r° = y, A = 0, I = 0; 
Repeat 

1 = 1 + 1; 

match step: 

h l = ^T r i-i. 

identify step: 

A ; = A'" 1 U {arg max.; \h l (j)\}; 
update step: 

x l = arg min JB:auppW£A i \\y - *z|| 2 ; 
r l = y — &x l ; 
Until stop criterion satisfied; 
Output: x k . 



In the Zth iteration, the inner products between the columns 
of <I> and the residual r 1 ^ 1 are calculated, and the index of 
the largest absolute value of inner products is added to A. 
Here, the residual r 1 ^ 1 from the former iteration represents 
the component of the measurement vector y that cannot be 
spanned by the columns of 3? indexed by A. In this way, the 
columns of 3? which are "the most relative" to y are iteratively 
chosen. The OMP algorithm is described in Table U 

In fact, OMP can be well expressed using y, A', Moore - 
Penrose pseudoinverse, and orthogonal projection operator. A 
brief review of them is given as follows (see [23| for more 
details). 

Let it I a denote the |A| x 1 vector containing the entries of 
u indexed by A. Define u(j) as the jth entry of vector u. 
Let 3? a denote the m x |A| matrix obtained by selecting the 
columns of sensing matrix <I> indexed by A. If 3? a has full 
column rank, then 3> A = (<I> a <I>a) _1 $a * s me Moore-Penrose 
pseudoinverse of <J>a- Let Pa = 4*a*a an ^ = I ~ Pa 
denote the orthogonal projection operator onto the column 
space of 3>a and its orthogonal complement, respectively. 
Define A a — Pjt& and A a = * when A = 0, then A a 
has the same size as From the theory of linear algebra, 
any orthogonal projection operator P obeys P = P T = P 2 
and the columns of A a indexed by A are zeros. 

In the kh iteration, we begin with the estimation A' -1 from 
the previous iteration. The discussion below demonstrates the 
generation of A'. 

In the update step of the previous iteration, which is actually 
solving a least square problem, one has 

r 1 - 1 =y $x l 1 =y fc^-i*]^!/ = P^y. (5) 

In the matching step, one has 

h l = ^V- 1 = * T (P I 7_ 1 ) T Pii- 1 2/ = Al^r 1 - 1 . (6) 

From ©, ©, and the fact that the columns of Aa indexed 
by A are zeros, it can be derived that 

h l (j) = 0, VjeA'" 1 . (7) 

Therefore arg max,,- \h l (j)\ A' -1 , |A'| = I. 

It is important to notice that the above property still holds 
when y and $ are replaced by the contaminated y and <&. To 
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see this, it is calculated that 

r '-l = y - $^-1 = y - Sa,-!*^,.^ = P^- iy , (8) 

h l = = * T (P A ^_ 1 ) T ^-iy = Al^r 1 - 1 , (9) 

where and A\ are defined by the perturbed sensing matrix 
4>. Due to the fact that the columns of A A ;-i indexed by 
A' -1 still equal zeros, (Q holds in the completely perturbed 
scenario. 

B. The Restricted Isometry Property (RIP) 

For each integer fc = 1,2, ... ,n, the RIP for any matrix 
A £ C mxn defines the restricted isometry constant (RIC) 5k 
as the smallest nonnegative number such that 

(l-5 k )\\x\\ 2 2 <\\Axf 2 <(l + 6 k )\\x\\ 2 2 (10) 

holds for any fc-sparse vector x 11391 . It is easy to check that if 
A satisfies the RIP of order k\ and k 2 with isometry constants 
8 kl and 5 k2 , respectively, and k\ < k 2 , then one has 6 kl < 5 k2 . 

Since its introduction, the RIP has been widely used as 
a tool to guarantee successful sparse recovery for various 
algorithms. For example, for the (No) process, the RIP of 
order 2k with S 2k < 0.03/ %/log fc guarantees exact recovery 
for any fc-sparse signal via ROMP algorithm fl5l ; the RIP of 
order 3fc with 5^ < 0.165 permits SP algorithm to exactly 
recover any fc-sparse signal ifTTl . 

However, analyzing the performance of OMP with RIP 
was relatively elusive before Davenport and Wakin's work 
in (23]. They demonstrated that RIP can be used for a 
very straightforward analysis of OMP in (No) process. It 
is shown that if y = and a; is a fc-sparse signal, then 
5k+i < l/(3\/fe) is sufficient for exact recover of OMP 
|23| (Th.3.1). Later, Liu and Temlyakov relaxed the bound 
to 1/((1 + V2)Vk) d (Th.5.2). Huang and Zhu further 
changed the bound to 1/(1 + 2yk), and they also discussed 
the performance for the (N\) process l25l . In ll27l . it has 
been proved that 1/(1 + \/k) is sufficient for (No) process, 
while for any given k > 1, there exists a sensing matrix with 
5k+i — l/Vk and a fc-sparse signal that exact recovery via 
OMP is not guaranteed. Therefore, if one uses the RIP of order 
k + 1 as a sufficient condition for exact recovery of a sparse 
signal via OMP, little improvement is possible. In terms of 
the number of measurements, it was demonstrated in 1 23 1 that 
5k+i < 1/(1 + Vre) requires 0(k 2 log(n/fc)) measurements, 
and the number is roughly the same as what is required by 
coherence-based analysis in ll2D . 

C. Assumptions and Notations 

A vector x £ C™ is fc-sparse if it contains no more than 
fc nonzero entries. Throughout this paper, however, the signal 
to be recovered is not limited to a sparse one. For a non- 
sparse signal x, define the 'head' x^ £ C™ as the fc-sparse 
signal that contains the fc largest magnitude entries of x, i.e. 
the best fc-sparse approximation of x, and define the 'tail' 
x^ = x — x^\ In order to delineate the compressibility of 
a general signal x, define 

\\x^h ||*(% 

P ||* (1) ||2' 7 7fc||x(l)|| 2 ' 



In this paper, x is assumed to be almost sparse, i.e. /3 and 7 
are far less than 1. When x^ = 0, one has /3 = 7 = 0, and 
x reduces to a sparse signal. 

The notation of strong-decaying sparse signals is introduced 
by Davenport and Wakin in [23 1. In our work, such concept 
is extended to general signals termed strong-decaying signals. 
Let {x(rrij)} 1<: i <n denote the entries of x rearranged in de- 
scending order by magnitude, x is called an a-strong-decaying 
signal if for all j £ {l,2,...n — 1} and x(m,j + i) 7^ 0, 
|a;(r7ij)|/|a;(mj + i)| > a, where a > 1 is a constant. 

When (N 2 ) or (N 2 ) process is concerned, it is necessary 
to consider the nature of b and E, and how they influence 
the process of OMP. This leads to the following definitions 
of relative bounds, which were introduced by Herman and 
Strohmer in l36l . 

The symbols \\-\\ 2 and || • ||a denote the spectral norm of a 
matrix and the largest spectral norm taken over all fc-column 
submatrices, respectively. The perturbations b and E can be 
quantified as 

II&II2 <p \\E\\i k) 
ll**l|a 

where ||<&a;||2, H^Hh, and H^H^ are nonzero. These relative 
upper bounds provide an access to analyze the influence of b 
and E, even though the exact forms of them are unknown. 
Throughout this paper, it is appropriate to assume that Eb and 
e are far less than 1. 

III. Contributions 

In this section, a completely perturbed scenario in the form 
of (f3]l is considered and the performance of OMP in (N 2 ) 
process is studied. Theorem 1 presents the RIP-based condition 
under which the support set of the head of x can be exactly 
recovered. In Theorem 2, we construct a sensing matrix and 
perturbations with which an almost sparse signal cannot be 
recovered. The RIC of the matrix is slightly bigger than that in 
the condition of Theorem 1, which indicates that the sufficient 
condition in Theorem 1 is rather tight. Several extensions with 
respect to special signals such as strong-decaying signals are 
put forward in Theorem 3 and 4. In Theorem 5, perturbations 
in the form of (01 is considered and the performance of OMP in 
(N 2 ) process is studied. The following theorems and remarks 
summarize the main results. 

Theorem 1: Suppose that the inputs y and $ of OMP al- 
gorithm are contaminated by noise in the form of OJ, and that 
the original signal x is almost sparse. Define the relative per- 
turbations Eh and e as in (fTTT i. Let to = min^g^pp^ti)) |cc(j)|, 
and 

1 23 

e h = ^(£ + e 6 + (1 + £„)(/? + 7))||z (1) || 2 . (12) 

If <l> satisfies the RIP of order fc + 1 with isometry constant 

S k +i <Q{k,£ h /t ), (13) 

then OMP will recover the support set of x^ exactly from 
y and $ in fc iterations, and the error between x^> and the 
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recovered fc-sparse signal x can be bounded as 



In (fl~3T > the function Q(-,-) is defined as 

1 3 



Q{u,v) 



fU+1 Ju+l 



(14) 



(15) 



Proof: The proof is postponed to Section IV-B. ■ 
Remark 1: Theorem 1 reveals that if the RIC of the available 
sensing matrix 3? is known to be under a threshold, it is 
guaranteed that the support set of the head of a signal can 
be recovered. 

It is of great significance to properly interpret Eh Ao in dT3T >. 
On one hand, the effects of b and E are reflected in terms 
of the worst-case relative perturbation Eb and e, respectively. 
Therefore, Eh represents a worst-case effect from perturbed y 
and €>. If more information on b and E is known, it may be 
possible to estimate a smaller Eh- On the other hand, to is the 
smallest magnitude of nonzero entries in x^> and represents 
the capability of a sparse signal to be recovered against 
perturbations. Therefore, t /sh has a natural interpretation as 
a lower bound on the minimum component SNR. One can see 
that the larger to/eh is, the weaker the requirement of 4+i 
needs. 

Remark 2: Considering (iVo) process, Theorem 1 general- 
izes the results in [23 1 — [25 J, ll27l . If vector y and matrix 3? 
are unperturbed, and x is fc-sparse, then (l3[ reduces to 



4+i < 



l 



Vk + r 



(16) 



which is exactly the result in [27 1 . 

Remark 3: It needs to be pointed out that in order to be 
well defined, Q(k,£h/to) should be greater than zero. Thus 
one gets 

to > 3e h . (17) 

It means that for the head of an almost sparse signal, the 
lower bound on the minimum component SNR should be large 
enough, so that its support can be extracted despite various 
noises. 

When x is fc-sparse and only the measurement vector y is 
perturbed, two corollaries can be derived from Theorem 1. 

Corollary 1: Suppose that E = in (0 and that the original 
signal x is fc-sparse. Let 

e h = 1.23£r b ||ac|| 2 - 

If <1? satisfies the RIP of order k + 1 with isometry constant 

S k+ i<Q(k,e h /t ), (18) 

then OMP will recover the support set of x exactly from y and 
in k iterations, and the error between x and the recovered 
fc-sparse signal x can be bounded as 



\x - as U < 



(19) 



Corollary 1': Suppose that E = in (O and that the 
original signal x is fc-sparse. If 3? satisfies the RIP of order 
k + 1 with isometry constant 

l-/x 



4+1 < rr ■, 
Vfc+ 1 

where fi g (0, 1) is a constant, and 

ll&Ha < M*o/3, 



(20) 



(21) 



then OMP will recover the support set of x exactly from y and 
<I> in k iterations, and the error between x and the recovered 
fc-sparse signal x can be bounded as 

l&lla 



as — as a < 



(22) 



Vi^4 

Remark 4: Both Corollary 1 and Corollary 1' concerns the 
condition for exact recovery of supp(a;) under measurement 
perturbation, but they are obtained from different point of 
views. In Corollary 1', the bound of 4+1 is a constant, while 
the 1% norm of measurement perturbation should be under 
a threshold. A comparison of Corollary 1' with a similar 
conclusion [25 1 (Th.2) will be given in Section V. 

When neither the measurement vector nor the sensing 
matrix is perturbed, the following corollary gives a sufficient 
condition under which the support of the head of an almost 
sparse signal can be exactly recovered. 

Corollary 2: Suppose that b = 0, E = in 0J, and that 
the original signal x is almost sparse. Let 

e ft = 1.23(/3 + 7 )||a ; ( 1 )||2. (23) 
If $ satisfies the RIP of order k + 1 with isometry constant 

4+1 < Q{k,e h /t ), (24) 

then OMP will recover the support set of x^ exactly from 
y and €> in k iterations, and the error between x^> and the 
recovered fc-sparse signal x can be bounded as 



\x-x [1) h 



Eft 



- -7r=r- (25) 

vl - 4 

Inspired by the work E7l . the following theorem reveals 
how tight the RIP-based condition in Theorem 1 is. 

Theorem 2: Consider the completely perturbed scenario (0). 
For any given positive integer k > 2, constants to > and 
< e < to, there exist an almost sparse signal x € C fe+1 , a 
sensing matrix €> £ c(fc+i)x(fc+i) ^ perturbations E and b such 
that the smallest nonzero magnitude of fc-sparse head asW is 
to, 

e= ||e|| 2 = \\$x^ - Ex + b\\ 2l 

and the perturbed sensing matrix $ satisfies the RIP of order 
k + 1 with isometry constant 



5 < 1 ! 
+ ~ \fk k to 



(26) 



Furthermore, OMP fails to recover the support set of x^ from 
y and 4> in k iterations. 

Proof: The proof is postponed to Section IV-C. ■ 
Compared with Theorem 3.2 in [[27), Theorem 2 takes 
general perturbation as well as non-sparseness of x into 
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consideration. Setting e = in Theorem 2, it reduces to the 
result in |27ll , thus our work is a generalized version. 

Remark 5: It will be shown that the bound ( fT3l is rather 
tight. Recall that the proof of Theorem 1 is twofold. First, d35| > 
is proved to be a sufficient condition for support recovery of 
a:' 1 ). Second, ||e||2 is estimated by ||e||2 < £h- Correspond- 
ingly, we demonstrate that little improvement can be made 
on them. First, comparing d26l l with (l35l l. these two bounds 
are both linear decreasing function of e/to, and as fc tends to 
infinity, the ratio of their y-intercepts approaches 1 while the 
ratio of their slopes approaches 3. Second, as for the upper 
bound of ||e||2, Proposition 3.5 in [16] and triangle inequality 
are used. For the sake of briefness, we assume that x is k- 
sparse and prove that 



1 - 

can be satisfied. First, let E 
signal x that satisfies ||<&£c||2 = 
Then it holds that 



ii vl + 4, ... ., 
|e||a = — — (e + e 6 )||a;||2 



= — g<&, an d choose a fc-sparse 
yl + 8k\\x\\2. Let b = e b $x. 



Ex 



(e + £b) 



\{e + e b )&x\ 



||*as||a y/l + h 



(e + e 6 )||x|| 



l-e l-e 

Due to the above two reasons, we show that the bound ([TJt 
in Theorem 1 is rather tight. 

For a-strong-decaying signals, the requirement of isometry 
constant 4+i can be relaxed, and the locations can even be 
picked up in the order of their entries' magnitude as long 
as the decaying constant a is large enough. This is what the 
following two theorems reveal. 

Theorem 3: Suppose that the inputs y and <& of OMP 
algorithm are contaminated by noise as in OJ, and that the 
original signal x is a-strong-decaying. Let 



£h 



1.23 
l-e 



(e + e b + (1 + e b )Ca 



and k* — (J2iZo a *) 2 /(£i=o a2l )> where C is a constant 
depending only on a. If <fr satisfies the RIP of order k + 1 
with isometry constant 



S k +i < Q(h*,e h /t ), 



(27) 



then OMP will recover the support set of x^ exactly from 
y and <I> in k iterations, and the error between x^ 1 ' and the 
recovered fc-sparse signal x can be bounded as 



\x-x^\\ 2 



< 



Vi - 4 



(28) 



Proof: The proof is postponed to Section IV-D. ■ 
Remark 6: Theorem 3 reveals that relaxed requirement of 
Sk+i guarantees the recovery of a strong-decaying signal, and 
that the larger a is, the easier the requirement of 4+1 can be 
satisfied. To see this, notice that for fc > 1, Cauchy-Schwarz 
inequality implies fc* < fc, and thus 

Q{k,e h /t )<Q{k*,e h /t ). 



Define fc* = L(a) = (£i=o a 1 ) 2 / Y^Zq ■ Because L(a) 
is a decreasing function of a, the larger a is, the smaller fc* 
is, and the easier the requirement of <5fc+i can be satisfied. 

Corollary 3: Suppose that the measurement vector and 
sensing matrix are unperturbed, and that the original signal 
x is a fc-sparse a-strong-decaying one. If $ satisfies the RIP 
of order fc + 1 with isometry constant 



Sk+i < 



1 



VF+i' 



(29) 



then OMP will recover x exactly from y and 4> in fc iterations. 

Remark 7: Because fc* < (a + l)/(a - 1), (|29]l can be 
replaced by 



4+i < 



l 



V(a + l)/(a-l) + l 



Furthermore, if a is far greater than 1, the requirement ap- 
proximately reduces to 5k+i < 0.5, which is very interesting. 

Theorem 4: Suppose that the inputs y and $ of OMP 
algorithm are contaminated by noise as in (0, and that the 
original signal x is a-strong-decaying. Let 



E/i = 



1.23 
l-e 



(e + e b + (1 + e b )Ca k )\\x 



where C is a constant depending only on a. If <fr satisfies the 
RIP of order fc + 1 with isometry constant 

x 1 2£h 
°k+l < - - -5—, 



and 



where 



a > max{G(4+i),1.2} 



G(u) 



l-3u 



2sh 
to 



(30) 



(31) 



then OMP will recover the support set of a;' 1 ' exactly from y 
and $ in fc iterations, and the recovery is in the order of the 
signal entries' magnitude. 

Proof: The proof is postponed to Section IV-E. ■ 
Remark 8: For (JVq) process with fc-sparse signal x, Dav- 
enport and Wakin proved that if $ satisfies the RIP of order 
fc + 1 with 5k+i < 1/3, and 



a > 



1 + 4+1(2^^1-1) 
1 - 34+i 



= I(Sk+i), 



(32) 



then OMP will recover x sequentially from y and $ in fc 
iterations [23] (Th.4.1). When x is no longer sparse, and the 
sensing matrix as well as the measurement vector is perturbed, 
Theorem 4 shows that the elements of supp(cc) can still be 
picked up sequentially. Specially, the following Corollary 4 is 
derived. 

Corollary 4: Suppose that y and $ are unperturbed, and 
that the original signal a; is a fc-sparse a-strong-decaying one. 
If $ satisfies the RIP of order fc + 1 with isometry constant 



4+1 < 



3' 
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and 



a > max 



1 + 5, 



k+l 



1 - 34+i 



1.2 



(33) 



then OMP will recover x exactly from y and 3? in k iterations, 
and the recovery is in the order of the entries' magnitude. 

For k > 1, one has 2y/k — 1 — 1 > 1. Thus, it can be 
seen from (l32l and ( T33l > that Corollary 4 is better than the 
conclusion in [23] when I(5k+i) is greater than 1.2. 

At the end of the main contribution, perturbations in the 
form of (@]i is considered. 

Theorem 5: Suppose that the inputs y and $ of OMP 
algorithm are contaminated by noise as in (|4]i, and that 
the original signal x is almost sparse. Define the relative 
perturbations e as that in (fTTT l. and e b as: 



< s b . 



Let 



E h = 1.23(e + e b + ee b + (1 + e 6 )(l + e)(0 + 7 ))||a; (1) || 2 . 
If $ satisfies the RIP of order k + l with isometry constant 

4+i < Q(k,e h /t ), 

then OMP will recover the support set of a^ 1 ) exactly from 
y and <fr in k iterations, and the error between x^ 1 ' and the 
recovered fc-sparse signal x can be bounded as 



\x-x^\\ 2 



< 



Proof: The proof is postponed to Section IV-F. ■ 
Remark 9: The definition of e b in Theorem 5 is different 
from that in Theorem 1 . This is due to the fact that e b denotes 
the relative measurement perturbation added to the output of 
the system, and the output in this scenario is $>x other than 
$>x. By comparison of Theorem 1 and 5, it can be seen that 
their main difference comes from the respective definition of 
e b - Based on the completely perturbed scenario several 
results similar to Theorem 2-4 can be derived without much 
difficulty. However, they are not included for conciseness. 



IV. Proofs 



A. Lemmas 



Before the proofs of the main theorems, two helpful lemmas 
are given first. Their proofs are postponed in Appendix. 

Lemma 1: Let {xi}i<i<z denote / positive variables satis- 
fying Xi/xi-i > a for all i, where a > 1 is a constant. Then 
the function f{x x , x 2 , ■ ■ ■ , Xi) = Q^ =1 x\ )/(Ei=i x if 
equals its minimum Q3;=q a2t ) I (J2i=o a *) 2 wnen — 
a, i — 2, ■ ■ ■ , 

Lemma 2: Suppose that the inputs y and $ of OMP 
algorithm are contaminated by noise as in (01, and that the 
original signal x is an a-strong-decaying one. Let 

l23 : (e + e b + (l + e b )Ca- k )\\x^\\ 2 . 



£h 



1 



For the Zth iteration, define x* G C" as the signal that contains 
the entries of x indexed by supp^ 1 ) ) \ A' -1 with the rest set 



to zeros. If $ satisfies the RIP of order k + l with isometry 
constant 4+i> one nas 



\ti(j)-x*(j)\ < 



Sk+i\\x*\\ 2 + e h 
i - 4+i 



(34) 



for all j 4 A' . 



B. Proof of Theorem 1 

Proof: First of all, it will be proved that OMP exactly 
recovers the support set of a^ 1 ) in k iterations. This proof 
consists of three parts. First, we prove that 

1 3 



implies 



4+i < 



4+i < 



J^b 

Vk + l Vk + 1 t 

1 VF+2 ||e|| 2 



(35) 



(36) 



W + 1 W + 1 W t 

for all 1 < k < k. Second, define e = Qx^ — Ex + b. 
We prove that d36l l is a sufficient condition for the support 
recovery in the Zth iteration with k = k — I + 1. At last, an 
upper bound of ||e||2 is given. 
First, define 

2 + VW 2- 



ci = 



C2 = 



(37) 



(1 + Vk)V¥t ' ' {l + Vk')Vk't 
then it's easy to check that c\ < c 2 - According to (135V it can 



be derived that 



4+i < 



l 



II II ^ C2 

- ci e 2 < — 



1 



Vk+1 _ ci\\/A; + l 

1 _ VF+2 \\e\\ 2 

W + 1 VV + i V¥ t ' 



- c lH e l|2 



(38) 



which implies 

The proof of the second part works by induction. To begin 
with, consider the first iteration where A = 0. (O indicate 
that 



y 



(* - E)x + b = + e. 



(39) 



Then, 



h 1 = * T y = * T (*cc (1) +e), 
which can be rewritten as 

h\i) = (*e 2 ,*a; (1) +e>, 

where (•, •) denotes the inner product in Euclidean space and 
6j denotes the ith natural basis. Define 

H= max Ih 1 ^)] 

iGsuppfajC 1 ) ) 

and U = (^x^\ ^x^ + e). On one hand, 

U = ^2x^\i)h\i) < \\x^\\xH < Vk\\x w \\ 2 H. (40) 
On the other hand, 



t/> ||*a; (1) ||i-||*a: (1) ||2||e|| 2 



> (1 - 6 k+1 )\\xW\\ 2 2 - ^l + 4+i||* (1) || 2 ||e| 



2- 



(41) 
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Thus one has 

H > -L((l - 4 + i)||* (1) || 2 - yJl + S k+1 \\e\\ a ). (42) 

For i ^ supp(a;( 1 )), Lemma 2.1 in [ 30 1 implies that 

\h 1 {i)\=\(*e i ,*xM) + (*e i ,e)\ 
<^ + i||^ (1) ||2 + ||*e i || a ||e|| 2 

<Sk+i\\x w h + yJl + S k +i\\eh. (43) 

Because ||a? (1) ||2 > Vkto, (ESJ of k' = k together with d42j 
and d43l indicate that 

H > \h\i)\, Vi £ supp(;c«), 

which guarantees the success of the first iteration. 

Now consider the general induction step. In the kh iteration, 
suppose that all previous iterations succeed, which means that 
A' -1 is a subset of supp^ 1 )). Define z 1 ^ 1 = x^ — x 1 ^ 1 , 
then supp(;z' _1 ) C supp^ 1 )). Because 

h l = <f> T (y - ^x 1 ' 1 ) = <t> T (*a; (1) - $x l 1 + e), 

one has 

ti(i) = {Qe^&z 1 - 1 +e). 

Define 

H= max \h!(i)\ 

iGsuppfajC 1 ) ) 

and U = ^z 1 ^ 1 + e). According to (0, it can be 

derived that 

U = J2z l ~ 1 tt)h l (i)< II^UppCxWM'-lli^ 
< Vk - I + l\\z l -%H = WWz 1 - 1 ^. (44) 

Following the steps in the proof for the first iteration, and 
noticing that ||.z i_1 ||2 > Vre to> it can t> e derived from (l36l l 
that 

H > \h l (i)\, Vi i supp(a;( 1 '). 

According to @, = for i G A' -1 , which guarantees 

the success of the Ith iteration. The proof of induction is 
completed. 

Thirdly, an upper bound of ||e||2 is given as follows. 
According to Proposition 3.5 in [16|, 

||^ (2) ||2<V / 1+^(N (2) || 2 + ^^) 



^l + 4(/3 + 7)l|a: (1) l|2, 



and 



\Ex\\ 2 <\\ExW\\ 2 + \\ExW\ 



<ll^«(llxW|| a + i|xW|| a + J!^!i) 



Therefore, 
||e|| 2 < ||*x (2) || 2 + ll^lla + llblb 

< ||*x^||2 + ||^x|| 2 + e 6 ||*a!||a 

< (||*a;< 2 >|| a + \\Ex\\ 2 )(l + e b ) + etW^x^h 

< ^ + S ~" (e + e b + (1 + e h )(p + 7))H* (1) I|2. (45) 

1 — e 

Noticing that S k < l/(Vk + 1) < 0.5, one has ||e|| 2 < en- 
Therefore dl"3l implies d35l) . which guarantees the exact re- 
covery of supp^ 1 -*). 

To finish the proof, the recovery error is bounded as follows. 
Because A = supp^ 1 )) is exactly recovered, one has 

sb| a = ®{y = *] v (*a^ (1) |a + e) = *«| A + * A e. (46) 
Thus 

\\x~X^\\ 2 <\\& A \\ 2 \\e\\ 2 <^=. 

yl-dk 



C. Proof of Theorem 2 

Proof: First, we prove that there exist a fc-sparse signal 
ajW with to as its smallest nonzero entries' magnitude, a 



vector e € 



•<k+l 



satisfying ||e||2 = e, and a perturbed sensing 



matrix <fr with 



5 < 1 V^l e 
+ ~ Vk k t 



(47) 



such that OMP fails to recover the support set of x^ 1 ' from 
<£> and y = ffrcc' 1 ) + e in k iterations. Let 



Ikxk alfexl 

Oixfe b 



(48) 



where a — 5/Vk and b = \[\ — S 2 are two constants with 
S < l/y/k. Since 



(49) 



Ikxk alfcxi 
ali xfc a 2 k + b 2 1 ' 

it can be derived that the eigenvalues {A^}^ 1 of are 

At = 1, l<i<k-l, \ k = l-5, A fc+ i = 1 + 5. (50) 

Thus for <I>, its RIC of order k + 1 satisfies 

4+i = S. (51) 

Let a; (1) = (i o lixfc,0) T and e = (Oi xfe .e) T , then the 
perturbed measurement vector 

y = $x^ +e = (i lixfe,e) T . 

Set 

Vfc-(e/t ) v /fc-l + (eAo) 2 
k + (e/t ) 2 

then the matching vector h 1 = tol(fe+i)xi> which implies that 
OMP fails in the first iteration. It is easy to check that 



< 



l-E 



l + 4(l + /3 + 7)l|z (1) l 



. ^ 1 Vfc^T e 
o S —j= ; • 

Vk k t 
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Second, let * = I( k +i)x(k+i), ^ (2) = (Oixfe, e/2) T , b 
(0 lxfc ,e/2) T , then 

Okxk alfexi 

Oixfe b-1 

and e = <I>a:( 2 ) — Ex + b, which completes the proof of 
Theorem 2. 



D. Proof of Theorem 3 

Proof: The proof of Theorem 3 is similar to that of 
Theorem 1 . For the sake of briefness, some revisions are made 
based on the proof of Theorem 1 . 
First, define 



k -1 



k -1 



fe'* = (^a') 2 /(E a2i )' 

According to Lemma 1, for any a-strong-decaying and k - 
sparse signal u, it holds that ||tt||i < %/fc'*||u.||2- Therefore, 
d40l and d44t can be replaced by 



U < Wx^^H < ||a; (1) || 2 VFi/ 



and 

U < \\z 



lsupp(cc( 1 ))\A 1 



— ll a;(1) | S upp(x<i))\A ! -i||l^ 



< Vk^\\x^\ snpp{x(1))XAl -4 2 H < Vk^Wz^hH 
respectively. Further, since 



equation Q6j can be replaced by 

1 Vk^ + 2 ||e|| a 



4+i < 



(52) 



(53) 



'k'* + 1 Vk'* + 1 Vk'*t 

Since 1 < k * < k*, d52l can be inferred by 

A . 1 3 ||ella 
Ofe+i < —f= 7= — ■ 

Second, an upper bound of /3 + 7 can be given in terms of 
a as follows 

p | ^ l^(m fc+1 )|((E^o^ 2 ') 1/2 + S^ «-VVfc) 

\x(m k+1 )\ \jY!l =1 a 21 

_ (1 - a- 2 )- 1 ' 2 + (Vk{l - a- 1 ))- 1 
a(a 2k - l)V2( a 2 _ i)-i/2 

< CoT k , 

where C is a constant only related to a. Therefore, 

Vi + 4, 



e a < 



< 



1-e 
V 1 + 4 

1 -£ 



-(e + e b + (l + e b )(/? + 7 ))||^ (1) || 
(e + e 6 + (l + E 6 )Ca- fc )||xW|| 2 . 



Notice that 4 < l/(\/fc* + l) < 0.5, therefore ((27j guarantees 
exact recovery of supp^ 1 - 1 ). ■ 



E. Proof of Theorem 4 

Proof: By induction it will be shown that (f30b guarantees 
the order of recovery. For the ith iteration, suppose that 
all the locations recovered in the previous iterations are in 
order. Define x* as that in Lemma 2. It will be demonstrated 
that OMP will choose the largest entry of x* , i.e. x(m{). 
According to Lemma 2, 



\ti{j)-x*{j)\< 



5k+i\\x*\\ 2 + e h 



1 - 4+i 

It can be calculated from a > 1.2 that 



(54) 



1 



1 — a 



1 

< 1 + -. 



Thus. 



00 1 
\x*\\ 2 < |a;(m ( )|(y>- 2t ) 1/2 < \x(mi)\(l + -). (55) 



1=0 



Combining (l54l i and $55[ , one has 

\h l (mi)\ >|as(m,)|-A, 

\h l ( mj )\ <|aj(mj + i)| + A < \x(m{)\/a + A, 



where 



A 



1-5, 



j £ {1 + 1,1 + 2,..., n}, 



■(J fc+ i|aj(mi)|(l + -)+eh). 



It is easy to check that \h l (nii)\ is greater than \h (rrij)\ for 
j £ {I + 1, 1 + 2, . . . , n}, if (O is satisfied. ■ 

F. Proof of Theorem 5 

Proof: For the sake of briefness, we only need to make 
some revisions based on the proof of Theorem 1 . Noticing that 
the inputs are <I? and y = &x + b = Qx^ +&x^ + Ex + b, 
4+i and e in the proof of Theorem 1 need to be replaced by 
4+i and *a; (2 ) + Ex + b. 

Define e = <^x^ + Ex + b. An upper bound of ||e||2 is 
given as follows. According to Proposition 3.5 in |[T6l . 

||W 2 )|| 2 < v /IT4(||z (2) || 2 + ^^) 



v ^T^( / 3 + 7 )|| x ( 1 )|| 



and 



\Ex\\ 2 < \\Ex^\\ 2 + \\Ex^\\ 2 



<\\Et\\\x^\\ 2 + \\x 



(2)1 



= e v / l + 4(l + /3 + 7)ll^ (1) ll2. 

Therefore, 

||e|| 2 < ||W 2 >|| 2 + ||^|| 2 +||6|| 2 

< ||$a3 (2) || 2 + |i-Ea;||2 + £ 6 ||*a;||2 

< (||*xW|| a + \\Ex\\ 2 )(l + e b ) +e b \\<S>x^\\ 2 

< £h, 
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and 



\x-x^h < 



I*lll2||e|| a < 



V. Related Works 

In this section, Theorem 2 in [25 1 and Theorem 3.1 in l40l 
are compared with Corollary 1' and Corollary 2 in this work, 
respectively. 

In (25), it is proved that for (Nx) process, the support of a 
fc-sparse signal x can be recovered, provided that 

4+i < l/(l + (-v / 6 + 2)Vfe) 

and ||6||2 < 5 k Vkto- By a comprehensive comparison, it is 
shown that this conclusion is not better than Corollary 1'. 

First, let ^satisfy l/(l + (V6 + 2)Vfe) = (1 - ^)/(Vk+ 1), 
i.e., _ _ 



/' 



l + (\/6 + 2)v / F 



Consider the ratio of the required upperbound of ||6||2 in the 
result in l25ll to that in Corollary 1': 



5 k Vkt Q 



-(l + {V6 + 2)Vk)6 k . 



mW 3 V6- 

It can be concluded from 4 <4+i < l/(l + (V6 + 2)Vk) 
that r < 3/(\/6 + 1) < 1, which means that the requirement 
of \\b\\2 in Corollary V is more relaxed. 

Second, the requirement of 4+1 m Corollary 1' is more 
relaxed. Because \i in Corollary V is optional, it can be chosen 
small enough that 

1/(1 + (V6 + 2)V*) < (1-aO/(V* + 1). 

Despite the difference in requirements, the recovery errors 
given in Theorem 2 in l25l and Corollary 1' are the same, 
since these errors are both derived when the support set of the 
sparse signal is perfectly recovered. 

In 11401 . the main result concerns the error estimation for 
OMR It is proved that 

||x - OMP s xf 2 < 2||x|| 2 (a s {x) + 46(2 + \\og 2 S])\\x\\ 2 ) , 

where a; is a non-sparse signal we wish to recover, OMPs x 
is the estimated solution via OMP in the 5th iteration, <rs(x) 
is the £2 error between the best 5-term approximation of x 
and x, and 6 is the RIC of order 2S. This conclusion gives 
an upper bound on the error between the original signal and 
the estimated result of any iteration in OMP. 

The original signal to be recovered in [40 1 is non-sparse, 
and the inputs y and are assumed non-perturbed. Thus the 
result actually gives an upper bound on the error between x 
and OMPs x for (Nq) process. Set S = k, and this result can 
be written as 



\x — x\ 



< 2\\x\ 



x 



(2)1 



2+4(5; 



2fc (2+riog 2 fcDHa) 



In Corollary 2, the result is 



yr^4 



09 + 7)||* (1) 



(56) 



(57) 



Before comparison, it is worth mentioning that there are 
fundamental differences between the above two conclusions. 
First, the non-sparse signal considered in this paper is almost 
sparse, whereas the one in ll40l is arbitrary. Second, conditions 
that guarantee the support set recovery of the head of x is 
the main concern in Corollary 2, and based on the successful 
support recovery, an upper bound on the error is estimated. In 
the reference, however, the £2 error is directly given regardless 
of the support recovery. In some practical applications, recov- 
ering the support set other than the more accurate estimation 
is a fundamental concern 1351 . Il38l . 

Despite the differences, a comparison of their recovery error 
estimations is given as follows. Notice that it is really hard 
to demonstrate which result is better, since the result in BOll 
involves 62k which does not appear in our work. However, a 
condition with 82k involved is given under which (156b is not 
better than d57l i. From d5Tb one has 



-x\\l= 



X 



< 



1 + 0.5 



W + 1) 2 \\X^\\ 2 2 + \\X^\\ 



1 -0.5 V 
= (3(/3 + 7) 2 +/3 2 )|| : r (1) ||^ 
<4((3 + 1 ) 2 \\x^\\i. 

If S 2k > (P + 7) 2 /4, from g8) one has 

\\x-xg < 164 fe ||xW||| < 16S2k\\x\\l 



(58) 



(59) 



Compared with d56l ), d59l actually gives a tighter bound. 
Besides, the above requirement of 82k can be written in terms 
of k, i.e., 

4 fc > 1/(54*;). (60) 

Assume nontrivially that /5 + 7 ^ 0. Thus Isupp^ 1 ')! = k 
and || a^ 1 ) || 2 > ykto. According to (TJl and ( 1231 . one has 



/3 + 7 < 



to 



3.69||a;( 1 )| 



< 



3.69Vfc' 



Combining (|58]l, d60j and (|6B, it holds that 

\\x-x\\l < m2k\\x (1) \\l < I65 2k \\x 



(61) 



(62) 



VI. Conclusion 

In this paper, considering a completely perturbed scenario 
in the form of y = <&x + b and $ = $ + E, the 
performance of OMP in recovering an almost sparse signal, 
i.e. x = RoMp(y, <&,■••), is studied. 

Though exact recovery of the head of x is no longer 
realistic, Theorem 1 shows that exact recovery of its support 
via OMP can be guaranteed under suitable conditions. Based 
on RIP, such conditions involve the sparsity, the relative 
perturbations of y and €>, and the smallest nonzero entry 
of x. Furthermore, the error between the the head of x 
and the output x is estimated. This completely perturbed 
framework extends the prior work in non-perturbed scenarios. 
Furthermore, we construct a sensing matrix and perturbations 
with which an almost sparse signal cannot be recovered. The 
RIC of the matrix is slightly bigger than that in the sufficient 
condition of Theorem 1, which indicates that the condition is 
rather tight. 
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In addition, when x is an a-strong-decaying signal, several 
extensions of Theorem 1 are put forward. Theorem 3 reveals 
that the requirement in Theorem 1 can be weaker to guarantee 
the exact recovery of support. Theorem 4 demonstrates that if 
a is large enough, the support is picked up in the order of 
its entries' magnitude. This advantage is of great significance 
in practical scenarios, since the larger entries are often more 
important than others, and recovery in order indicates that the 
algorithm is more stable. In the end, Theorem 5 discussed the 
other scenario of general perturbations, which is in the form 
of y = &x + b and <fr = <fr + E, with the recovery process 
written as x = RompOj, 3>, ■ ■ • )■ Please notice that several 
results similar to Theorem 2-4 are available for (N 2 ) process, 
however, they are not included for conciseness. 

VII. Appendix 

A. Proof of Lemma 1: 

Proof: First of all, see {xi\i = 1, 2, • • • , / — m} as / — m 
constants, and define the function with variable x > axi- m 

l—m I— tit 

g( x ) = x i + bmx2 )/C^2 Xi + c ™ x ) 2 ' 

1=1 1=1 

where b m = YJv=o °< 2% ■• c ™ = S£=o &\ 1 < m < I. Then 
g(x) > g(axi- m ). 

The proof lies in the fact that g(x) can be written as 

/ \ b m 2/7 y 
9K X ) = ZT~ 



Therefore, it can be inducted that f(xx, ■ ■ ■ , Xi) is no less than 
f(xi, ■ ■ ■ , a l ~ 1 x 1 ), which concludes the proof. ■ 

B. Proof of Lemma 2: 

Proof: It can be concluded from ©, ©, and ( f39l > that 



A A i-ix* + i^^P^ie 
hi + h 2l 



(64) 



where hi = AT,_i A^i-ix*, h 2 — A T Al _ 1 P^ l _ 1 e. Because 
||jc*||o + |A j-1 | + 1 < fc + 1, according to Lemma 3.3 in 1231 , 
for all j A' -1 , it holds that 



\hiV)-x*m<T^T-\\**h- 

1 - Ofc+1 

According to Lemma 3.2 in [23], for j ^ A' -1 , 



(65) 



\h 2 (j)\\ =(A A i-ie j ,P£i- 1 e) < x/l + 6 k+ i\\P^ ie \\ 2 



< 



1 



i - 4+i 



e o < 



(66) 



i - 4+i 

Notice that the last inequality holds since ||e||2 < Sh, which 
has been given in the proof of Theorem 3. Combining d64b . 
and triangle inequality, one finally gets 

\h l (j)-x*(j)\<\hi(j)-x*(j)\+\h 2 (j)\ 



< 



where 



S k+ i\\x*\\ 2 +e h 
1 - 4+i 



v = — y, 



i=i 



, t — hl l — m l — m 

&{Yxi?-YA)/{—Yxi\ 



i=l 



i=l z=l i=l 

and y — x + 8. Because —6 + r]/b m > 0, g(x) equals its 
minimum when 

y° = -0 + v/bm, (63) 



which further infers that 

x°=y°-e = V /b m -20 = 



El — m 2 
i=l X i 



b T l ~ m X ■' 
u m Z_^i—i x z 

Because x° < a;;- TO < axi- m < x and g(x) is an increasing 
function when x > x°, g{x) > g(axi- m ). 

Lemma 1 is proved by induction. To begin with, let m = 
1 and fix {xi\i = 1, • • • ,1 — 1}, then the above conclusion 
implies f(xi, ■ ■ ■ ,xi-i,xi) > f(xi, ■ ■ ■ ,Xi-x,axi-x). 

Furthermore, assume that 

f(xx, ■ ■ ■ ,xi- m+ x, ■■■ ,xi) 
>f[xx, • • • , Xl-m+i, • • • , a m ~ 1 xi- m+ i). 
The above conclusion gives 

f(xi,--- , Xl-m, Xl-m+1, ■ ■ ■ ,a m ~ 1 Xl-m + l) 
— fi,^X i ? X[ — m , X, • • • x) |a;=aa;; _ m 



=f(xi, ■■■ ,xi. 



m j i 
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